System for integral analysis and management of video data

ABSTRACT

A system for integrated analysis and management of video data includes a routing server configured to receive video data from an external input video providing device, a video database configured to store the received video data, a video analyzing server cluster including a plurality of video analyzing servers configured to analyze the video data, a metadata database configured to store metadata of the video data, and a video search server configured to search the metadata database and the video database. As described in various embodiments of the present disclosure, the system can perform integrated analysis of various video data received from a number of channels and provide an analysis result, and can also rapidly provide an accurate search result in response to a user&#39;s request for search.

TECHNICAL FIELD

The present disclosure relates to a method and system for integratedanalysis and management of video data. Specifically, the presentdisclosure provides a method and system that can allocate a lot of videodata input from a plurality of channels, which provides input video, toa specific video analyzing server of a video analyzing server clusterincluding a plurality of video analyzing servers through a routingserver, recognize and analyze an object recorded in the video data, andprovide the analyzed video data to an external image output device, andcan also classify objects by category while recognizing images, extractmetadata of each object, store the extracted metadata in a database foreasy search later, and enable a lot of users to participate in deeplearning for object recognition, which enables more sophisticatedrecognition and distribution and utilization of object data learnedthrough deep learning.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted as prior art by inclusion in this section.

A CCTV security system that collects image information input throughfixed or mobile cameras installed in a certain area or place, an objectrecognition system that classifies individual objects by applyingintelligent image analysis, and a license plate recognition system forcollecting various types of image information about a vehicle, such asspeed of a driving vehicle, are known as general systems for collectinginformation from video data.

These systems only analyze specific information input from a recordingdevice for each purpose and provide the analyzed specific information toa user. Also, these systems have a certain limitation in the number ofrecording devices that provide input video data, i.e., channels toprocess, and, thus, it is difficult to add a large number of channels inparallel, and video data input simultaneously from a plurality ofchannels cannot be processed smoothly.

Meanwhile, in a system for processing video data input from a pluralityof channels, video generated by respective video recording devices hasdifferent characteristics and is recognized by different methods andalgorithms accordingly. Therefore, whenever a new video channel isadded, the structure of an image recognition and processing system needsto be modified.

Further, a multi-channel analyzing system that enables a user to easilyand quickly search for analyzed and recognized object data is required,and if a plurality of users participates in modeling through deeplearning in the analysis, a video analyzing system can be more reliable.

SUMMARY

The present disclosure is intended to solve the above-described problemsand/or various other problems.

More specifically, the present disclosure is intended to provide asystem that forms video data received from an input video providingdevice, which provides various types of input video, into a videodatabase for efficient search; allocates the video data to a specificvideo analyzing server among a plurality of video analyzing serversconfigured as a video analyzing server cluster according to the type orcharacteristics of the video to perform video analysis and recognitionand thus to increase recognition efficiency; provides the analyzed andrecognized video data in a video format satisfying the requirements of auser; and efficiently specifies a channel and time corresponding tovideo including a specific object according to a search query from auser device and quickly searches for the video from the video database.

Also, the present disclosure is intended to facilitate effectiveparticipation of many general users or deep learning experts in deeplearning for object recognition through a deep learning platform webinterface for a number of external users; evaluate and store a videoanalysis model or an image recognition algorithm generated through deeplearning; and enable them to be used usefully in other systems andapplied as independent applications with economic value.

A system for integrated analysis and management of video data accordingto an embodiment of the present disclosure may include: a routing serverconfigured to receive video data from an external input video providingdevice; a video database configured to store the received video data; avideo analyzing server cluster including a plurality of video analyzingservers configured to analyze the video data; a metadata databaseconfigured to store metadata of the video data; and a video searchserver configured to search the metadata database and the videodatabase. Herein, the routing server may be further configured toallocate analysis of the video data to a specific video analyzing serveramong the plurality of video analyzing servers of the video analyzingserver cluster according to characteristics of the received video data,and the characteristics of the received video data may include at leastone of a type of a channel that records the video data, a characteristicof the channel, and a type of an object to be recognized among objectsrecorded (or photographed) in the video data.

The specific video analyzing server allocated the analysis of the videodata by the routing server among the plurality of video analyzingservers of the video analyzing server cluster is further configured toanalyze the video data to recognize one or more objects included thereinand generate the metadata according to a recognition result. Thespecific video analyzing server may be further configured to generateinverse features based on the generated metadata, and the metadatadatabase may be further configured to store the generated inversefeatures. The video search server may be further configured to receive avideo search query from an external user device and search at least oneof the metadata database and the video database based on the receivedsearch query. Also, the video search server may be further configured toobtain information about video data required in the search query bysearching the metadata database based on the search query, and obtainthe video data required in the search query by searching the videodatabase based on the obtained information.

Meanwhile, the system may further include: a video converter configuredto generate a graphic user interface in a predetermined format based onan analysis result of the video analyzing server cluster; and an I/Oserver configured to transmit an analysis result image of the videoanalyzing server cluster to an external video output/control device andreceive a video search query from an external user device.

The system may further include: a deep learning server configured tolearn video data stored in the video database and generate a videoanalysis model; and a learning data result value database configured tostore the video analysis model generated in the deep learning server.Herein, the deep learning server may be configured to provide at leastone of easy training and hard training for video data learning to a userconnected through a deep learning platform web interface for a number ofusers outside the system.

Further, a method for integrated analysis and management of video datathat is performed by one or more computing devices in a video analysisand management system according to an embodiment of the presentdisclosure may include: a process of receiving video data from any onechannel of an input video providing device including a plurality ofchannels; a process of allocating the received video data to a specificvideo analyzing server of a video analyzing server cluster including aplurality of video analyzing servers in the video analysis andmanagement system; a process of analyzing the video data by the specificvideo analyzing server to recognize one or more objects includedtherein; and a process of generating a graphic user interface in apredetermined format based on an analysis result and an objectrecognition result. The method may further include a process of storingthe received video data in a video database in the video analysis andmanagement system, and the video database may be a NoSQL database basedon Hbase.

The method may further include: a process of generating metadata of therecognized one or more objects by the specific video analyzing server; aprocess of generating inverse features based on the generated metadataby the specific video analyzing server; and a process of storing thegenerated metadata and inverse features in a metadata database in thevideo analysis and management system. According to another embodiment ofthe present disclosure, the method may further include: a process ofreceiving a search query for a specific object from an external userdevice; a process of searching at least one of the metadata database andthe video database based on the received search query; and a process oftransmitting a search result to the external user device. In this case,the method may further include: a process of determining whether thesearch result is suitable for the search query; and a process ofsearching at least one of the metadata database and the video databasebased on the received search query when it is determined not suitablefor the search query.

In a computer-readable storage medium that stores a computer program forintegrated analysis and management of video data according to anembodiment of the present disclosure, instructions included in thecomputer program may include: at least one instruction to receive videodata from any one channel of an input video providing device including aplurality of channels; at least one instruction to allocate the receivedvideo data to a specific video analyzing server of a video analyzingserver cluster including a plurality of video analyzing servers in avideo analysis and management system; at least one instruction toanalyze the video data by the specific video analyzing server torecognize one or more objects included therein; and at least oneinstruction to generate a graphic user interface in a predeterminedformat based on an analysis result and an object recognition result.Herein, the computer program may further include: at least oneinstruction to receive a search query for a specific object from anexternal user device; at least one instruction to search at least one ofa metadata database and a video database based on the received searchquery; and at least one instruction to transmit a search result to theexternal user device.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative embodiments andfeatures described above, further embodiments and features will becomeapparent by reference to the drawings and the following detaileddescription.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features of the present disclosure will bedescribed in detail with reference to the accompanying drawings.Understanding that these drawings depict only several examples inaccordance with the disclosure and are, therefore, not to be consideredlimiting of its scope, the disclosure will be described with additionalspecificity and detail through use of the accompanying drawings, inwhich:

FIG. 1 is a configuration view of a system for integrated analysis andmanagement of video data according to an embodiment of the presentdisclosure;

FIG. 2 is a configuration view of a system for integrated analysis andmanagement of video data according to another embodiment of the presentdisclosure;

FIG. 3 is a configuration view of a system for integrated analysis andmanagement of video data according to yet another embodiment of thepresent disclosure;

FIG. 4A shows an example of metadata extracted from input video dataaccording to an embodiment of the present disclosure, and FIG. 4B showsan example of inverse features generated based on the metadata accordingto an embodiment of the present disclosure;

FIG. 5 shows an example of channel metadata for channels of an inputvideo providing device;

FIG. 6 shows a specific example of a picture displayed on a videooutput/control device as a video analysis result;

FIG. 7 shows an example of a process for integrated analysis andmanagement of video data according to at least some embodiments of thepresent disclosure;

FIG. 8 shows an example of a process for search of video data accordingto at least some embodiments of the present disclosure; and

FIG. 9 shows an example of a computer program product that can be usedfor integrated analysis and management of video data according to atleast some embodiments of the present disclosure.

DETAILED DESCRIPTION

The terms used herein are used only to describe specific examples, butdo not intend to limit the present disclosure. A singular expressionincludes a plural expression unless it is clearly construed in adifferent way in the context. All terms including technical andscientific terms used herein have the same meaning as commonlyunderstood by a person with ordinary skill in the art to which thisinvention belongs. It will be further understood that terms, such asthose defined in commonly used dictionaries, should be interpreted ashaving a meaning that is consistent with their meaning in the context ofthe specification and relevant art and should not be interpreted in anidealized or overly formal sense unless expressly so defined herein. Insome cases, even terms defined in the present disclosure should not beinterpreted as excluding embodiments of the present disclosure.

The foregoing features and other features of the present disclosure willbe sufficiently apparent from the following descriptions with referenceto the accompanying drawings. These drawings merely illustrate severalembodiments in accordance with the present disclosure. Therefore, theyshould not be understood as limiting the present disclosure. The presentdisclosure will be described in more detail with reference to theaccompanying drawings.

FIG. 1 illustrates a system for integrated analysis and management ofvideo data according to an embodiment of the present disclosure.

Referring to FIG. 1 , a system 100 for integrated analysis andmanagement of video data may be configured to receive input video froman external input video providing device 110 and process the inputvideo. System 100 may provide the processed video data to a videooutput/control device 160. Also, system 100 may receive an input of asearch query from a user device 162 and provide video data correspondingto the search query to user device 162 and/or video output/controldevice 160. Herein, user device 162 may be linked with videooutput/control device 160 or may operate separately from videooutput/control device 160.

A communication linkage between system 100 and input video providingdevice 110, input video providing device 110, video output/controldevice 160 and/or user device 162 may be established in various wired orwireless ways. A network that enables the communication linkage mayinclude, for example, a radio frequency (RF) network, a third generationpartnership project (3GPP) network, a long term evolution (LTE) network,a world interoperability for microwave access (WIMAX) network, theInternet, a local area network (LAN), a wireless LAN, a wide areanetwork (WAN), a personal area network (PAN), a Bluetooth network, anear field communication (NFC) network, a satellite broadcastingnetwork, an analog broadcasting network, and a digital multimediabroadcasting (DMB) network, but is not limited thereto.

Input video providing device 110 may include a plurality of input videogenerating devices. The input video generating device may include avideo recording device such as a CCTV. Also, the input video generatingdevice may be a device, such as a synthetic video generator, thatgenerates synthetic video instead of an actual image obtained through avideo recording device. Each of such various input video generatingdevices constitutes a single video channel, and input video generated byeach channel is provided as original video data to system 100.Hereinafter, each input video generating device of input video providingdevice 110 will also be referred to as a channel.

System 100 shown in FIG. 1 may include a routing server 120, a videoanalyzing server cluster 130, a video database 140, a video converter150, a metadata database 170, an I/O server 180 and a video searchserver 190. Herein, I/O server 180 may be omitted in some cases. In thiscase, video converter 150 may directly provide processed video data tovideo output/control device 160 through the above-describedcommunication linkage, and user device 162 may perform a video searchthrough a direct communication linkage with video search server 190.

Routing server 120 may receive input video from each channel of inputvideo providing device 110 and store the input video as original videodata in video database 140. The original video data stored in videodatabase 140 may be provided to user device 162 in response to a user'srequest for search.

Also, routing server 120 may route processing of images from a specificchannel to a specific server in video analyzing server cluster 130according to characteristics of each channel of input video providingdevice 110.

Video analyzing server cluster 130 is composed of a plurality of videoanalyzing servers, and each video analyzing server may be equipped withone or more high-specification GPUs for high-quality video analysis.Also, each video analyzing server may be designed to be suitable foranalysis of specific video. For example, each video analyzing server maybe classified into a video analyzing server suitable for recognizing andprocessing a person, a video analyzing server suitable for recognizingand processing a car, and the like depending on an object included invideo. Further, each video analyzing server may be designed to besuitable for processing video data of a specific situation, and may beclassified into, for example, a video analyzing server suitable forprocessing video data with generally low brightness, such as videorecorded through a camera installed in a tunnel, a video analyzingserver suitable for processing video data with generally highbrightness, such as video recorded through a camera installed outdoors,and the like. Furthermore, each video analyzing server may be designedto be suitable for processing video data according to characteristics ofeach channel, such as a type of the channel. For example, each videoanalyzing server may be classified into a video analyzing server with afixed channel suitable for processing high-quality video data, such as aCCTV, a video analyzing server with a mobile channel suitable forprocessing low-quality video data, such as a drone, and the like. Eachvideo analyzing server of video analyzing server cluster 130 may analyzevideo data from a specific channel allocated by the routing server,extract metadata of the video data and then store the metadata inmetadata database 170. Also, each video analyzing server may generateinverse features from the extracted metadata and store the inversefeatures in metadata database 170. Specific examples of the metadata andinverse features will be described later with reference to FIG. 4A andFIG. 4B.

Further, each video analyzing server of video analyzing server cluster130 may process input video by applying an image recognition algorithmsuitable for a channel for video data to be processed. In this case, thevideo analyzing server may retrieve metadata of a corresponding channelfrom a channel meta-database regarding attributes of each channel andprocess input video by applying an image recognition algorithm suitablefor the channel metadata. The channel metadata may include camera ID,camera IP, encoding type (e.g., H.264, H.265, etc.), camera type (e.g.,CCTV, drone, etc.), image resolution (e.g., HD, 4K, etc.), recordingdevice type (e.g., fixed, mobile, etc.), content category (e.g., parkinglot, downtown street, etc.), camera location, camera height, tilt angle,pan angle, number of decoding frames per second, use, and the like, butmay not be limited thereto. The channel meta-database, which is a set ofchannel metadata, may be stored in the form of metadata database 170 ora separate database, and may be searched and used by each videoanalyzing server of video analyzing server cluster 130. A specificexample of the channel metadata and the use thereof for video analysiswill be described later with reference to FIG. 5 .

Meanwhile, the input video from each channel recognized and analyzed byeach video analyzing server of video analyzing server cluster 130 isprovided to video converter 150, and video converter 150 converts therecognized and analyzed input video into a predetermined format suitablefor transmission to and display on video output/control device 160.Herein, the predetermined format may be previously set by a user ofvideo output/control device 160. The input video converted into thepredetermined format by video converter 150 may be provided to videooutput/control device 160 through I/O server 180. A specific example ofthe predetermined format and a picture displayed on video output/controldevice 160 will be described later with reference to FIG. 6 .

A user of user device 162 accessing system 100 may make a request forsearch for specific video data to system 100 through user device 162 byusing a specific search query. In an embodiment, the user of user device162 may specific an object to be searched or retrieved through thepicture displayed on video output/control device 160 and make a searchquery. The search query may include a tag or label of a specific object,a channel that provides video including the object, a place and/or timeperiod thereof.

I/O server 180 of system 100 may receive the search query from userdevice 162 and provide the search query to video search server 190. Ifthe above-described I/O server 180 is omitted, the search query fromuser device 162 may be directly provided to video search server 190.

Video search server 190 may firstly retrieve metadata database 170 byusing the search query transmitted from user device 162. In this case,video search server 190 may specify a search target object included inthe search query by using a tag, label or thumbnail of the object to besearched and obtains a channel that records video including the objectand a recording time of the video from metadata database 170. Videosearch server 190 may search video database 140 for original video databased on the obtained channel and recording time, and may provide thesearched original video data to user device 162 through I/O server 180.A specific example of searching metadata database 170 and video database140 and searching for and providing original video data suitable for asearch query from user device 162 by video search server 190 will bedescribed below in more detail with reference to FIG. 4A and FIG. 4B.

Meanwhile, video database 140 may be configured as a relationaldatabase, and may be configured as a NoSQL database in which the schemais not previously defined in some embodiments. According to anembodiment of the present disclosure, if video database 140 isconfigured as a NoSQL database, it may be stored in HBase. HBaseclassified as a column-oriented NoSQL database may store a number ofcolumns in a row, and video database 140 of the present disclosure maygenerate input video input from a specific channel in a row withoutlimitation to the number of columns by using this attribute of HBase.

As a non-limiting example, video database 140 may record input videoreceived from a specific channel in a row on a daily basis, and theinput video may be generated as an individual file every second in therow. In this case, a total of 86,400 (60 seconds×60 minutes×24 hours)files, i.e., files, may be generated in a row. If video database 140 isconfigured in this way, it is only necessary to search a row of aspecific date for video without a need to search all the rows.Accordingly, the search efficiency can be improved.

In system 100 shown in FIG. 1 , video data received from input videoproviding device 110 that provides various input video streams may beconfigured as video database 140 for efficient search, and may beallocated (or assigned) to a suitable video analyzing server of videoanalyzing server cluster 130 through routing server 120 according to atype or characteristics of input video, and, thus, the input video isanalyzed and recognized, which results in improvement in recognitionefficiency. Also, in system 100 shown in FIG. 1 , the analyzed andrecognized video data may be provided in a format satisfying therequirements of a user to video output/control device 160 through videoconverter 150, and metadata may be extracted from the video data forobject search and stored in metadata database 170 together with inversefeatures thereof, and, thus, a channel and time of video including aspecific object can be efficiently specified according to a search queryfrom user device 162 and can be quickly searched from video database140.

FIG. 2 is a configuration view of a system for integrated analysis andmanagement of video data according to another embodiment of the presentdisclosure.

The components shown in FIG. 2 perform identical or similar functions tothe respective components shown in FIG. 1 and are assigned the samereference numerals as the respective components shown in FIG. 1 .Hereinafter, the detailed description of the components that performidentical or similar functions to the respective components shown inFIG. 1 will be omitted for simplicity.

A system 200 for integrated analysis and management of video data shownin FIG. 2 is different from system 100 shown in FIG. 1 in that videoconverter 150 is omitted. That is, in FIG. 2 , video converter 150 isprovided on the user side outside system 200, i.e., in videooutput/control device 160 of the user.

Herein, video converter 150 may receive video data analyzed andrecognized by video analyzing server cluster 130 of system 200 throughI/O server 180. Video converter 150 may convert the analyzed andrecognized video data into a format previously set by the user of videooutput/control device 160 and then provide it to video output/controldevice 160, and video output/control device 160 may display the same.According to an embodiment, the format previously set by the user may bea screen or graphic user interface to be displayed. This graphic userinterface may include, for example, a part showing real-time images froma plurality of channels, a part showing the state of the spaces coveredby the respective channels, and a part showing the movement paths ofobjects recorded by each channel as an analysis result of the channel.The graphic user interface will be described in detail later withreference to FIG. 6 .

FIG. 3 is a configuration view of a system for integrated analysis andmanagement of video data according to yet another embodiment of thepresent disclosure.

The components shown in FIG. 3 perform identical or similar functions tothe respective components shown in FIG. 1 and are assigned the samereference numerals as the respective components shown in FIG. 1 .Hereinafter, the detailed description of the components that performidentical or similar functions to the respective components shown inFIG. 1 will be omitted for simplicity.

A system 300 for integrated analysis and management of video data shownin FIG. 3 further includes a deep learning server 310 and a deeplearning data result value database 320 in addition to system 100 shownin FIG. 1 .

Deep learning server 310 may analyze original video data stored in videodatabase 140 to generate a video analysis model or image recognitionalgorithm, and may store the generated video analysis model or imagerecognition algorithm as a deep learning result in deep learning dataresult value database 320. Further, the video analysis model or imagerecognition algorithm generated by deep learning server 310 may be usedby each video analyzing server of video analyzing server cluster 130.Each video analyzing server of video analyzing server cluster 130 maysearch deep learning data result value database 320 for a video analysismodel or image recognition algorithm suitable for a specific channelallocated to itself by routing server 120 and/or a specific object andmay retrieve it for application to video analysis and objectrecognition.

In an embodiment related to deep learning of original video data andgeneration of a result value video analysis model, deep learning server310 may divide and analyze the original video data stored in videodatabase 140 by predetermined categories. For example, deep learningserver 310 may analyze each object according to a type of the objectrecorded in original video data in consideration of characteristics andsurrounding conditions of the object, and may generate a video analysismodel or image recognition algorithm associated therewith. In anotherexample, deep learning server 310 may analyze an object according to achannel that records original video data in consideration of a type andcharacteristics of the channel, and may generate a video analysis modelor image recognition algorithm therefor. In this case, deep learningserver 310 may use metadata of the channel, and the metadata may includecamera ID, camera IP, encoding type (e.g., H.264, H.265, etc.), cameratype (e.g., CCTV, drone, etc.), image resolution (e.g., HD, 4K, etc.),recording device type (e.g., fixed, mobile, etc.), content category(e.g., parking lot, downtown street, etc.), camera location, cameraheight, tilt angle, pan angle, number of decoding frames per second,use, and the like, but may not be limited thereto. The channelmeta-database, which is a set of channel metadata, may be stored in theform of a separate database, and may be used to generate a videoanalysis model or image recognition algorithm by deep learning server310.

The video analysis model or image recognition algorithm generated bydeep learning server 310 may be stored by category in deep learning dataresult value database 320. For example, deep learning data result valuedatabase 320 may store image recognition algorithms for person, car,pet, etc. depending on a type of an object, image recognition algorithmsfor street, park, parking lot, etc. depending on positionalcharacteristics of a channel, and image recognition algorithms for CCTV,drone, etc. depending on a type of a channel.

The video analysis model or image recognition algorithm stored in thedatabase 320 as a deep learning model automatically learned internallyfor specific data may include a video analysis model or imagerecognition algorithm generated externally and added in the form of aplug-in in addition to the video analysis model or image recognitionalgorithm generated by deep learning server 310. The externallygenerated video analysis model or image recognition algorithm isgenerated by a deep learning server outside system 300, and may begenerated by using video data from input video providing device 110and/or learning separate video data irrelevant thereto. Since theexternally generated video analysis model or image recognition algorithmis added, the image analysis and recognition rate of each videoanalyzing server of video analyzing server cluster 130 can be furtherimproved.

Meanwhile, the video analysis model or image recognition algorithmgenerated by deep learning server 310 may be provided to other systemsoutside system 300. This video analysis model or image recognitionalgorithm is generated by deep learning server 310 analyzing a largeamount of video data from input video providing device 110, and may beused usefully in the other systems and applied as an independentapplication with economic value.

As shown in FIG. 3 , a deep learning platform web interface 330 for anumber of users may be further provided outside system 300, and deeplearning platform web interface 330 enables many general users or deeplearning experts participating in deep learning to participate inlearning through deep learning server 310 of the system. Deep learningplatform web interface 330 may be configured as a web server, but is notlimited thereto.

Users may access deep learning platform web interface 330 through theirown separate devices, and the users may include general users such asthe user of video output/control device 160 and/or the user of userdevice 162. Also, the users who access deep learning platform webinterface 330 may be deep learning experts with specialized knowledge ofdeep learning. Deep learning server 310 may provide hard training to thedeep learning experts and may provide easy training to the generalusers. In the case of easy training, deep learning server 310 mayprovide several options that a user can easily select through deeplearning platform web interface 330, and the user may participate indeep learning by simply selecting an option. The options provided bydeep learning server 310 to deep learning platform web interface 330 foreasy training may include, for example, an option to select a targetobject, an option to select an image type, an option to select a deeplearning model, an option to select a learning dataset, etc., but maynot be limited thereto. Dep learning server 310 is configured to learnabout video data from video database 140 according to the optionselected by the user through deep learning platform web interface 330.

Deep learning server 310 may provide labeling for learning objects. Forlabeling, deep learning server 310 may allow a labeling operator toperform labeling through deep learning platform web interface 330. Inother words, the labeling operator may access deep learning server 310through deep learning platform web interface 330 and may performlabeling for an object in video data being analyzed and learned, andlabeling for objects may be reflected in the video analysis model orimage recognition algorithm and stored together in deep learning dataresult value database 320. Also, deep learning server 310 may provide anenvironment in which a plurality of labeling operators can accesssimultaneously and work simultaneously.

Further, deep learning server 310 may evaluate the generated videoanalysis model or image recognition algorithm and/or the labeling forobjects. Feedback from each video analyzing server of video analyzingserver cluster 130 may be used for this evaluation. The evaluationresults may also be stored together in deep learning data result valuedatabase 320 and may be used later for evaluation and distribution of avideo analysis model or image recognition algorithm correspondingthereto. Although deep learning server 310 shown in FIG. 3 is configuredas a single server, it may be configured as a deep learning servercluster, such as video analyzing server cluster 130, composed of aplurality of deep learning servers in some embodiments. In this case, arouting device may be needed to allocate original video data from videodatabase 140 to a suitable deep learning server according tocharacteristics of the corresponding channel, and this process may beperformed by routing server 120 or a non-illustrated separate routingdevice. Further, a routing device may be needed to allocate operationsof a plurality of users who accesses through deep learning platform webinterface 330 to a specific deep learning server among a plurality ofdeep learning servers in the deep learning server cluster, and thisprocess may be performed by I/O server 180 or a non-illustrated separaterouting device.

System 300 shown in FIG. 3 is further equipped with deep learning server310 and deep learning data result value database 320 that stores a videoanalysis model or image recognition algorithm as a result of learning,and, thus, the efficiency and performance related to video analysis andobject recognition of each video analyzing server of video analyzingserver cluster 130 can be improved and a plurality of users canparticipate in learning of deep learning server 310 or perform objectlabeling through deep learning platform web interface 330 for aplurality of external users. Meanwhile, deep learning server 310 isconfigured as a cluster of a plurality of deep learning servers, and,thus, a more accurate video analysis model or image recognitionalgorithm can be generated by a deep learning server suitable for eachchannel and/or video through parallel analysis and learning. As aresult, the accuracy and efficiency in video analysis and objectrecognition of the video analyzing servers included in video analyzingserver cluster 130 can be further improved.

FIG. 4A shows an example of metadata extracted from input video dataaccording to an embodiment of the present disclosure, and FIG. 4B showsan example of inverse features generated based on the metadata accordingto an embodiment of the present disclosure.

As described above with reference to FIG. 1 , each video analyzingserver of video analyzing server cluster 130 may analyze video data of aspecific channel allocated from the routing server, extract metadata ofthe video data, and then store the metadata in metadata database 170,and FIG. 4A shows an example of such metadata. Also, each videoanalyzing server of video analyzing server cluster 130 may generateinverse features from the extracted metadata and store them metadatadatabase 170, and FIG. 4B shows an example of such inverse features.

Referring to FIG. 4A, the metadata may be classified into a person P, acar C, etc. depending on a type of an object recognized from video dataprovided by a corresponding channel for each of a plurality of channelsCH1, CH2, . . . . That is, objects included in the video data recordedby each of channels CH1 and CH2 may include various objects such aspersons and cars, and in the metadata shown in FIG. 4B, objectsrecognized by each channel are listed in a row according to the type ofobject. Accordingly, in the row corresponding to a person object P ofCH1, metadata of persons recognized in the video data of CH1 are listedin each column, and in the row corresponding to a car object C of CH2,metadata of cars recognized in the video data of CH1 are listed in eachcolumn. Similarly, metadata of persons recognized in the video data ofCH2 are listed in each column of the corresponding row, and metadata ofcars recognized in the video data of CH2 are listed in each column ofthe corresponding row.

In the column of each row of metadata, the recognized objects are listedin chronological order from left to right. Also, each video analyzingserver of video analyzing server cluster 130 may perform tagging toassign a unique tag to each recognized object and may put the same tagto objects recognized as the same. For example, in the row of the personobject P of CH1, a tag of P0138 is tagged to a person recognized firstin the video data of CH1 and a tag of P2413 is tagged to a personrecognized subsequently. Then, a person recognized further subsequentlyis recognized as the same person as the previously recognized persontagged with P0138 and thus is tagged with the same tag of P0138.

As for each row of metadata shown in FIG. 4A, an image of a recognizedobject is firstly stored and information about the image (e.g., ahatched area on the right of “P0138 image” in the row of the personobject P of CH1) is stored right after the image. The information aboutthe image, i.e., the object, includes a tag (e.g., P0138) of the object,a timestamp indicating the time at which the video was recorded,information about the corresponding channel, and the location of theoriginal video data (e.g., the storage location in video database 140),but is not limited thereto.

FIG. 4B shows inverse features generated based on the metadata shown inFIG. 4A. As the inverse features, information about a channel thatrecords a corresponding object is listed in chronological order based oneach object. Referring to FIG. 4B, the inverse features are broadlyclassified into objects, such as a person P and a car C, and a personobject P is further classified into individual person objects P0138,P2413, . . . and a car object C is further classified into individualcar objects C1217, C3448, . . . . In each object row, for example, aninverse feature row of the object P0138, channels CH1 and CH2 thatrecord the object P0138 are followed by detailed information indicatedas a dotted area. Herein, the detailed information includes informationabout the location of the corresponding channel, a timestamp indicatingthe time at which the corresponding object was recorded, and thelocation of the original video data in which the corresponding objectwas recorded, but is not limited thereto.

The metadata and inverse features generated by each video analyzingserver of video analyzing server cluster 130 and stored in metadatadatabase 170 may be used to search for video of a specific object inresponse to an external user's request for search. As described abovewith reference to FIG. 1 , the user of user device 162 accessing fromthe outside of system 100 may make a request for a search for specificvideo data to system 100 through user device 162 by using a specificsearch query, and the search query may include a tag of a specificobject. For example, if the user wants to see original video of a personcorresponding to a tag of the object P0138 recognized through videooutput/control device 160, the user may transmit a search queryincluding the tag P0138 to system 100 through user device 162. Videosearch server 190 of system 100 that receives the query may firstlysearch for the inverse features stored in metadata database 170 as shownin FIG. 4B. As a result of the search, video search server 190 mayobtain information about channels that record or capture the objectP0138, the time at which the object was recorded or captured by eachchannel and/or the storage location of the original video data. Videosearch server 190 may search video database 140 based on the informationabout the original video data of the object P0138 and retrieve theoriginal video data to provide the same to the user through videooutput/control device 160 or to a separate display device connected touser device 162.

Herein, the user's search query may be provided in various forms asrequired by the user, and video search server 190 of system 100 may usethe metadata and/or inverse features stored in metadata database 170 anddata stored in video database 140 in various ways in order to providethe optimal search query for the user's requirements in response tovarious queries of the user.

FIG. 5 shows an example of channel metadata for channels of an inputvideo providing device. Input video providing device 110 described abovewith reference to FIG. 1 may include devices, i.e., channels, thatgenerate and/or provide input video in various forms. Examples of thechannels may include a CCTV, a drone, etc. depending on a type of achannel, and may include a general camera, a thermal imaging camera, adepth camera, etc. depending on a photographing method. Also,characteristics of a channel may be classified depending on the locationof the channel and the installation height and angle of the channel.Since input video is processed by applying image recognition algorithmdata suitable for respective channels of various types, it is possibleto obtain a higher recognition rate and faster recognition performance.

FIG. 5 shows an example of channel metadata of two case channels, i.e.,case 1 and case 2, consisting of 13 categories. A channel ID is shown ina first row, and an IP address of a camera is shown in a second row.Channels may be classified by the encoding type of a channel, and can beclassified by camera type, image resolution, recording device type andcontent category. The channels can be further classified by location andheight of a camera, tilt angle, pan angle, number of decoding frames persecond, use of a channel. An example of channel metadata of each casechannel classified as described above is shown in FIG. 5 . The channelmeta-database, which is a set of channel metadata, may be stored in theform of metadata database 170, deep learning data result value database320 shown in FIG. 3 or a separate database.

In an embodiment, after allocated analysis of a specific channel byrouting server 120, each video analyzing server of system 100 describedabove with reference to FIG. 1 may retrieve a video analysis model orimage recognition algorithm suitable for channel metadata of the channelwith reference to the channel metadata of the channel and perform videodata analysis and object recognition based on the video analysis modelor image recognition algorithm. The video analysis model or imagerecognition algorithm may be generated through learning in deep learningserver 310 described above with reference to FIG. 3 and stored in deeplearning data result value database 320, or may be provided from theoutside and stored in deep learning data result value database 320.

FIG. 6 shows a specific example of a picture displayed on a videooutput/control device as a video analysis result.

As described above with reference to the embodiment shown in FIG. 1 ,input video recognized and analyzed by each video analyzing server ofvideo analyzing server cluster 130 is provided to video converter 150,and video converter 150 converts the recognized and analyzed input videointo a predetermined format suitable for transmission to and display onvideo output/control device 160. Also, as described above with referenceto the embodiment shown in FIG. 2 , when video converter 150 is providedon the user side outside system 200, i.e., in video output/controldevice 160 of the user, video converter 150 receives video data analyzedand recognized by video analyzing server cluster 130 of system 200through I/O server 180 and converts the analyzed and recognized videodata into a format previously set by the user of video output/controldevice 160. FIG. 6 shows an example of a picture, i.e., a graphic userinterface 600, converted into a predetermined format by video converter150 and displayed on video output/control device 160.

Referring to FIG. 6 , graphic user interface 600 displayed on videooutput/control device 160 includes a channel video display part 610 thatdisplays real-time video from a plurality of channels, a space displaypart 620 that displays the state of the spaces covered by the respectivechannels, a channel analysis part 630 that displays an analysis resultof each channel and an object movement path analysis part 640 thatdisplays the movement paths of objects recognized by each channel. Eachof these parts is listed and configured as an example, and some of theseparts may be removed or other necessary screen parts may be addedaccording to the requirements and needs of the user of videooutput/control device 160.

Channel video display part 610 displays the state where a video analyzerserver analyzes video recorded by each of a plurality of channels inreal time and recognizes objects. In the embodiment shown in FIG. 6 , anobject to be recognized is limited to a person. Channel video displaypart 610 displays images from four channels, i.e., CH1, CH2, CH3 andCH4. CH1 shows an image of an entrance, CH2 shows an image of acorridor, and each of CH3 and CH4 shows an image of a meeting room. Inthe images from CH1 and CH2, any person as an object to be recognized isnot recognized, and in each of the images from CH3 and CH4, threepersons are recognized.

Space display part 620 displays the installation locations of therespective channels and the present locations of objects recognized bythe channels on a floor plan of the space installed with the channels.That is, CH1 is installed by the entrance on the floor plan of thespace, CH2 is installed at the corridor, CH3 is installed at a meetingroom 1, and CH4 is installed at a meeting room 2. Also, as shown in FIG.6 , three objects, i.e., persons, tagged with P03, P04 and P06 arelocated in the meeting room 1 installed with CH3, and three personstagged with P01, P02 and P05 are located in the meeting room 2 installedwith CH4.

Object movement path analysis part 640 displays movement paths of therespective objects. Graphic user interface 600 shown in FIG. 6 displaysthe present state of objects P01 to P06 located in the meeting room 1and the meeting room 2 through respective movement paths. For example,object P04 of object movement path analysis part 640 is recognizedsequentially by each of CH1, CH2, CH3, CH2 and CH3, which means that theperson tagged with P04 enters the entrance (CH1), passes through thecorridor (CH2), gets into the meeting room 1 (CH3), gets out again tothe corridor (CH2) and presently enters again the meeting room 1 (CH3).Also, object P01 of object movement path analysis part 640 is recognizedsequentially by CH1 and CH4, which means that the person tagged with P01enters the entrance (CH1), passes through the corridor (CH2), gets intothe meeting room 2 (CH4), but is not recognized by CH2 installed at thecorridor.

Channel analysis part 630 displays a recognition result of each channel.As for the number of times of recognition of each object by each channelshown in object movement path analysis part 640, each of CH1, CH3 andCH4 performs object recognition a total of four times and CH2 performsobject recognition a total of eight times. Channel analysis part 630graphically displays the number of times of object recognition, whichmakes it convenient for the user to see the number of times of objectrecognition by each channel.

In this way, video converter 150 receives analysis and objectrecognition result data as well as a video signal of each channel fromvideo analyzing server cluster 130, generates graphic user interface 600in a format previously set by the user of video output/control device160 and provides graphic user interface 600 to video output/controldevice 160. Thus, it is possible to provide the user with the presentstate of objects in a space and an analysis result in a more intuitiveand comprehensive manner.

FIG. 7 shows an example of a process 700 for integrated analysis andmanagement of video data according to at least some embodiments of thepresent disclosure.

Process 700 shown in FIG. 7 may include one or more operations,functions or actions as illustrated by blocks 710, 720, 730, 740, 750,760, 770 and/or 780. The operations schematically illustrated in FIG. 7are provided by way of example only, and some of the operations may beoptional, may be combined into fewer operations, or may be expanded toadditional operations without departing from the spirit of the disclosedembodiment. The blocks of process 700 shown in FIG. 7 may have identicalor similar functions or actions to those described above with referenceto FIG. 1 to FIG. 3 , but are not limited thereto. Further, in anon-limiting embodiment, process 700 shown in FIG. 7 may be performed bysystem 100 shown in FIG. 1 , system 200 shown in FIG. 2 , or system 300shown in FIG. 3 .

Therefore, the blocks of process 700 shown in FIG. 7 will be describedbelow in association with the components used in the systems shown inFIG. 1 to FIG. 3 .

Referring to FIG. 7 , process 300 may begin in block 710 to receiveinput video from a channel of an input video providing device.

In block 710, system 100, 200 or 300 for integrated analysis andmanagement of video data receives input video generated by a pluralityof input video generating devices included in input video providingdevice 110. Input video providing device 110 may include a plurality ofinput video generating devices. The input video generating device may bea video recording device such as a CCTV, a camera installed in a drone,or a device, such as a synthetic video generator, that generatessynthetic video instead of an actual image obtained through a videorecording device. Each of such various input video generating devicesconstitutes a single video channel, and input video generated by eachchannel is provided as original video data to system 100, 200 or 300.Process 700 may continue from block 720 to block 720 to allocate thereceived input video to a specific video analyzing server of a videoanalyzing server cluster and/or to block 730 to add the received inputvideo to a video database.

In block 720, routing server 120 of system 100, 200 or 300 may allocatethe input video received from input video providing device 110 to aspecific video analyzing server of video analyzing server cluster 130.In this case, routing server 120 may route processing of images from aspecific channel to a specific server in video analyzing server cluster130 according to characteristics of each channel of input videoproviding device 110. Each video analyzing server of the video analyzingserver cluster 130 may be designed to be suitable for analysis ofspecific video, and routing server 120 may allocate the video to asuitable video analyzing server according to channel characteristics ofthe input video in a state in which characteristics of each videoanalyzing server are identified in advance. Meanwhile, in block 730,routing server 120 of system 100, 200 or 300 may receive the input videofrom each channel of input video providing device 110 and the inputvideo as original video data in video database 140. The original videodata stored in video database 140 may be provided to user device 162 inresponse to a user's request for search. Process 700 may continue fromblock 720 to block 740 to analyze the input video and recognize objectsincluded in the video.

In block 740, each video analyzing server of video analyzing servercluster 130 of system 100, 200 or 300 may analyze the input video fromchannel allocated by routing server 120 to perform object recognition.In this case, each video analyzing server may process the input video byapplying an image recognition algorithm suitable for a channel for videodata to be processed. Further, each video analyzing server may retrievemetadata of a corresponding channel from a channel meta-databaseregarding attributes of each channel and process the input video byapplying an image recognition algorithm suitable for the channelmetadata. Furthermore, each video analyzing server may perform videoanalysis and object recognition by using a video analysis model or imagerecognition algorithm generated through learning of deep learning server310 and stored in deep learning data result value database 320 or avideo analysis model or image recognition algorithm generated by anexternal system and stored in deep learning data result value database320. Process 700 may continue from block 740 to block 750 to generate ascreen output image in a predetermined format based on a video analysisresult data and/or to block 770 to generate metadata of the recognizedobject.

In block 750, video converter 150 of system 100 or 300 generates agraphic user interface in a predetermined format suitable fortransmission of the recognized and analyzed input video to videooutput/control device 160 and display of the same thereon. In this case,the generated graphic user interface may have the same configuration asgraphic user interface 600 shown in FIG. 6 , but is not limited thereto.Meanwhile, when video converter 150 is provided in video output/controldevice 160 outside the system as in system 200 shown in FIG. 2 , block750 and its subsequent block 760 may be omitted. In this case, system200 may transmit the input video subjected to analysis and objectrecognition by video analyzing server cluster 130 and related data toexternal video converter 150 through I/O server 180 or without passingthrough I/O server 180. Process 700 may continue from block 750 to block760 to transmit the generated screen output image to a videooutput/control device.

In block 760, I/O server 180 of system 100 or 300 may transmit thescreen output image generate by video converter 150 in the same form asgraphic user interface 600 shown in FIG. 6 to video output/controldevice 160. As described above, I/O server 180 may be omitted in system100 or 300, and in this case, video converter 150 may transmit thegenerated screen image directly to video output/control device 160without passing through any I/O server. Video output/control device 160provided outside the system may display and provide the received screenimage to the user.

In block 770, each video analyzing server of video analyzing servercluster 130 in system 100, 200 or 300 may generate metadata of eachobject through analysis and object recognition. As shown in FIG. 4A, themetadata may be classified in rows such as a person P, a car C, etc.depending on a type of an object recognized from video data provided bya corresponding channel for each of a plurality of channels CH1, CH2, .. . , and may be composed of an object image of objects recognized fromeach row as well as information about the object image, such as a tag ofthe object, a timestamp, information about the corresponding channel,and the location of the original video data listed in chronologicalorder. Process 700 may continue from block 770 to block 780 to generateinverse features from the metadata.

In block 780, each video analyzing server of video analyzing servercluster 130 in system 100, 200 or 300 may generate inverse features fromthe metadata generated in block 770. As shown in FIG. 4B, the inversefeatures may be composed of information about a channel that records acorresponding object listed in chronological order based on each object.

The metadata generated in block 770 and the inverse features generatedin block 780 may be stored in metadata database 170 of system 100, 200or 300 and may be used later to search for video of a specific object inresponse to an external user's request for search.

Meanwhile, the operation of generating the metadata in block 770 and thesubsequent operation of generating the inverse features in block 780 maybe performed in parallel with block 750 where the screen output image isgenerated in the predetermined format based on the video analysis resultdata or may be performed before or after block 750.

FIG. 8 shows an example of a process for search of video data accordingto at least some embodiments of the present disclosure.

A process 800 shown in FIG. 8 may include one or more operations,functions or actions as illustrated by blocks 810, 820, 830, 840 and/or850. The operations schematically illustrated in FIG. 8 are provided byway of example only, and some of the operations may be optional, may becombined into fewer operations, or may be expanded to additionaloperations without departing from the spirit of the disclosedembodiment. The blocks of process 800 shown in FIG. 8 may have identicalor similar functions or actions to those described above with referenceto FIG. 1 to FIG. 3 , but are not limited thereto. Further, in anon-limiting embodiment, process 800 shown in FIG. 8 may be performed bysystem 100 shown in FIG. 1 , system 200 shown in FIG. 2 , or system 300shown in FIG. 3 . Therefore, the blocks of process 800 shown in FIG. 8will be described below in association with the components used in thesystems shown in FIG. 1 to FIG. 3 .

Referring to FIG. 8 , process 800 may begin in block 810 to receive asearch query from an external user device.

In block 810, system 100, 200 or 300 for integrated analysis andmanagement of video data may receive a search query requesting videodata for a specific object from external user device 162. In anembodiment, the user of user device 162 may write a search query byspecifying an object to be searched through a screen displayed on videooutput/control device 160. The search query may include a tag or labelof a specific object, a channel that provides video including theobject, a place and/or time period thereof. Process 800 may continue toblock 820 to search a metadata database based on the search queryreceived in block 810 and obtain the location of original data.

In block 820, video search server 190 of system 100, 200 or 300 mayobtain the storage location of original video data suitable for thesearch query by searching for metadata stored in metadata database 170based on the search query received from external user device 162. Inthis case, video search server 190 may specify a search target objectwith a tag, label or thumbnail of the object included in the searchquery, and may obtain the location of video data including the objectand/or a channel that records the video data and a recording time frommetadata database 170. Process 800 may continue to block 830 to searchfor the video from the video database the corresponding video from avideo database based on the location of the original data obtained inblock 820.

In block 830, video search server 190 of system 100, 200 or 300 maysearch video database 140 based on the location of the video datarequested by the search query and/or the channel that records the videodata and the recording time obtained in block 820. Video database 140may be a NoSQL database based on Hbase. In this case, video database 140may generate input video input from a specific channel in a row withoutlimitation to the number of columns. When video search server 190obtains the channel and recording time suitable for the search queryfrom metadata database 170, it is only necessary to search a row of thechannel and recording time without a need to search all the rows.Accordingly, it is possible to search for video data much faster.Process 800 may continue to block 840 to verify whether the videosearched in block 830 is suitable for the search query.

In block 840, video search server 190 of system 100, 200 or 300 mayverify whether the video data searched from video database 140 issuitable for the search query from external user device 162. Thisverification operation may be performed in various known ways, and insome embodiments, the verification operation of block 840 may beomitted. As a result of the verification, if it is determined that thesearched video data is not suitable for the search query, video searchserver 190 may perform block 820 again to search metadata database 170,or in some embodiments, video search server 190 may perform block 830 tosearch video database 140 without performing the operation of searchingmetadata database 170. Process 800 may continue to block 850 to transmitthe video searched in block 840 to the external user device.

In block 850, I/O server 180 of system 100, 200 or 300 may transmitvideo data searched by video search server 190 and suitable for thesearch query to external user device 162. In some examples of the systemin which I/O server 180 is omitted, video search server 190 may directlytransmit a search result to external user device 162. Further, in someembodiments, video converter 150 may generate an appropriate graphicuser interface in a format previously set by the user based on thesearch result and transmit the graphic user interface to user device 162and/or the video output/control device. Process 800 shown in FIG. 8enables an external user to be easily and quickly search for and view anoriginal video image of a specific object that he or she wants to findand to be provided with an object analysis result such as a movementpath of the object.

FIG. 9 shows an example of a computer program product 900 that can beused for integrated analysis and management of video data according toat least some embodiments of the present disclosure. The computerprogram product is provided by using a signal bearing medium 910. Insome embodiments, one or more signal bearing mediums 910 of computerprogram product 900 may include a computer-readable medium 930 and/or arecordable medium 940.

Instructions 920 included in signal bearing medium 910 may be executedby one or more computing devices included in system 100, 200 or 300 forintegrated analysis and management of video data. Instructions 920 mayinclude at least one of an instruction to receive input video from achannel of an input video providing device, an instruction to allocatethe received input video to a specific video analyzing server of a videoanalyzing server cluster, an instruction to add the received input videoto a video database, an instruction to analyze the input video andrecognize objects included in the video, an instruction to generatemetadata of the recognized object, an instruction to generate inversefeatures from the generated metadata, an instruction to generate ascreen output image in a predetermined format based on a video analysisresult data, and an instruction to transmit the generated screen outputimage to a video output/control device.

The above description of the present disclosure is provided for thepurpose of illustration, and it would be understood by a person withordinary skill in the art that various changes and modifications may bemade without changing technical conception and essential features of thepresent disclosure. Thus, it is clear that the above-describedembodiments are illustrative in all aspects and do not limit the presentdisclosure. For example, each component described to be of a single typecan be implemented in a distributed manner. Likewise, componentsdescribed to be distributed can be implemented in a combined manner.

The claimed subject matter is not limited in scope to the particularimplementations described herein. For example, some implementations maybe in hardware, such as employed to operate on a device or combinationof devices, for example, whereas other implementations may be insoftware and/or firmware. Likewise, although claimed subject matter isnot limited in scope in this respect, some implementations may includeone or more articles, such as a signal bearing medium, a storage mediumand/or storage media. This storage media, such as CD-ROMs, computerdisks, flash memory, or the like, for example, may have instructionsstored thereon, that, when executed by a computing device, such as acomputing system, computing platform, or other system, for example, mayresult in execution of a processor in accordance with the claimedsubject matter, such as one of the implementations previously described,for example. As one possibility, a computing device may include one ormore processing units or processors, one or more input/output devices,such as a display, a keyboard and/or a mouse, and one or more memories,such as static random access memory, dynamic random access memory, flashmemory, and/or a hard drive.

There is little distinction left between hardware and softwareimplementations of aspects of systems; the use of hardware or softwareis generally (but not always, in that in certain contexts the choicebetween hardware and software can become significant) a design choicerepresenting cost vs. efficiency tradeoffs. There are various vehiclesby which processes and/or systems and/or other technologies describedherein can be effected (e.g., hardware, software, and/or firmware), andthat the preferred vehicle will vary with the context in which theprocesses and/or systems and/or other technologies are deployed. Forexample, if an implementer determines that speed and accuracy areparamount, the implementer may opt for a mainly hardware and/or firmwarevehicle; if flexibility is paramount, the implementer may opt for amainly software implementation; or, yet again alternatively, theimplementer may opt for some combination of hardware, software, and/orfirmware. The foregoing detailed description has set forth variousembodiments of the devices and/or processes via the use of blockdiagrams, flowcharts, and/or examples. Insofar as such block diagrams,flowcharts, and/or examples contain one or more functions and/oroperations, it will be understood by those within the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, several portions of the subject matterdescribed herein may be implemented via Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signalprocessors (DSPs), or other integrated formats. However, those skilledin the art will recognize that some aspects of the embodiments disclosedherein, in whole or in part, can be equivalently implemented inintegrated circuits, as one or more computer programs running on one ormore computers (e.g., as one or more programs running on one or morecomputer systems), as one or more programs running on one or moreprocessors (e.g., as one or more programs running on one or moremicroprocessors), as firmware, or as virtually any combination thereof,and that designing the circuitry and/or writing the code for thesoftware and or firmware would be well within the skill of one of skillin the art in light of this disclosure. In addition, those skilled inthe art will appreciate that the mechanisms of the subject matterdescribed herein are capable of being distributed as a program productin a variety of forms, and that an illustrative example of the subjectmatter described herein applies regardless of the particular type ofsignal bearing medium used to actually carry out the distribution.

While certain example techniques have been described and shown hereinusing various methods and systems, it should be understood by thoseskilled in the art that various other modifications may be made, andequivalents may be substituted, without departing from claimed subjectmatter. Additionally, many modifications may be made to adapt aparticular situation to the teachings of claimed subject matter withoutdeparting from the central concept described herein. Therefore, it isintended that claimed subject matter not be limited to the particularexamples disclosed, but that such claimed subject matter also mayinclude all implementations falling within the scope of the appendedclaims, and equivalents thereof.

Throughout this document, the term “connected to” may be used todesignate a connection or coupling of one element to another element andincludes both an element being “directly connected to” another elementand an element being “electronically connected to” another element viaanother element. Through the whole document, the term “on” that is usedto designate a position of one element with respect to another elementincludes both a case that the one element is adjacent to the otherelement and a case that any other element exists between these twoelements. Further, through the whole document, the term “comprises orincludes” and/or “comprising or including” used in the document meansthat one or more other components, steps, operation and/or existence oraddition of elements are not excluded in addition to the describedcomponents, steps, operation and/or elements unless context dictatesotherwise. Through the whole document, the term “about or approximately”or “substantially” is intended to have meanings close to numericalvalues or ranges specified with an allowable error and intended toprevent accurate or absolute numerical values disclosed forunderstanding of the present disclosure from being illegally or unfairlyused by any unconscionable third party.

The scope of the present disclosure is defined by the following claimsrather than by the detailed description of the embodiment. It shall beunderstood that all modifications and embodiments conceived from themeaning and scope of the claims and their equivalents are included inthe scope of the present disclosure.

1. A system for integrated analysis and management of video data,comprising: a routing server configured to receive video data from anexternal input video providing device; a video database configured tostore the received video data; a video analyzing server clusterincluding a plurality of video analyzing servers configured to analyzethe video data; a metadata database configured to store metadata of thevideo data; and a video search server configured to search the metadatadatabase and the video database.
 2. The system for integrated analysisand management of video data of claim 1, wherein the routing server isfurther configured to allocate analysis of the video data to a specificvideo analyzing server among the plurality of video analyzing servers ofthe video analyzing server cluster according to characteristics of thereceived video data.
 3. The system for integrated analysis andmanagement of video data of claim 2, wherein the characteristics of thereceived video data include at least one of a type of a channel thatrecords the video data, a characteristic of the channel, and a type ofan object to be recognized among objects recorded in the video data. 4.The system for integrated analysis and management of video data of claim1, wherein the video database is a NoSQL database based on Hbase.
 5. Thesystem for integrated analysis and management of video data of claim 1,wherein the specific video analyzing server allocated the analysis ofthe video data by the routing server among the plurality of videoanalyzing servers of the video analyzing server cluster is furtherconfigured to analyze the video data to recognize one or more objectsincluded therein and generate the metadata according to a recognitionresult.
 6. The system for integrated analysis and management of videodata of claim 5, wherein the specific video analyzing server is furtherconfigured to generate inverse features based on the generated metadata,and the metadata database is further configured to store the generatedinverse features.
 7. The system for integrated analysis and managementof video data of claim 1, wherein the video search server is furtherconfigured to receive a video search query from an external user deviceand search at least one of the metadata database and the video databasebased on the received search query.
 8. The system for integratedanalysis and management of video data of claim 7, wherein the videosearch server is further configured to obtain information about videodata required in the search query by searching the metadata databasebased on the search query, and obtain the video data required in thesearch query by searching the video database based on the obtainedinformation.
 9. The system for integrated analysis and management ofvideo data of claim 1, further comprising: a video converter configuredto generate a graphic user interface in a predetermined format based onan analysis result of the video analyzing server cluster.
 10. The systemfor integrated analysis and management of video data of claim 1, furthercomprising: an I/O server configured to transmit an analysis resultimage of the video analyzing server cluster to an external videooutput/control device and receive a video search query from an externaluser device.
 11. The system for integrated analysis and management ofvideo data of claim 1, further comprising: a deep learning serverconfigured to learn video data stored in the video database and generatea video analysis model; and a learning data result value databaseconfigured to store the video analysis model generated in the deeplearning server.
 12. The system for integrated analysis and managementof video data of claim 11, wherein the deep learning server is furtherconfigured to provide at least one of easy training and hard trainingfor video data learning to a user connected through a deep learningplatform web interface for a number of users outside the system.
 13. Amethod for integrated analysis and management of video data that isperformed by one or more computing devices in a video analysis andmanagement system, comprising: a process of receiving video data fromany one channel of an input video providing device including a pluralityof channels; a process of allocating the received video data to aspecific video analyzing server of a video analyzing server clusterincluding a plurality of video analyzing servers in the video analysisand management system; a process of analyzing the video data by thespecific video analyzing server to recognize one or more objectsincluded therein; and a process of generating a graphic user interfacein a predetermined format based on an analysis result and an objectrecognition result.
 14. The method for integrated analysis andmanagement of video data of claim 13, further comprising: a process ofstoring the received video data in a video database in the videoanalysis and management system, wherein the video database is a NoSQLdatabase based on Hbase.
 15. The method for integrated analysis andmanagement of video data of claim 14, further comprising: a process ofgenerating metadata of the recognized one or more objects by thespecific video analyzing server; a process of generating inversefeatures based on the generated metadata by the specific video analyzingserver; and a process of storing the generated metadata and inversefeatures in a metadata database in the video analysis and managementsystem.
 16. The method for integrated analysis and management of videodata of claim 15, further comprising: a process of receiving a searchquery for a specific object from an external user device; a process ofsearching at least one of the metadata database and the video databasebased on the received search query; and a process of transmitting asearch result to the external user device.
 17. The method for integratedanalysis and management of video data of claim 16, further comprising: aprocess of determining whether the search result is suitable for thesearch query; and a process of searching at least one of the metadatadatabase and the video database based on the received search query whenit is determined not suitable for the search query.
 18. The method forintegrated analysis and management of video data of claim 13, whereinthe process of analyzing the video data by the specific video analyzingserver to recognize one or more objects included therein includes: aprocess of analyzing the video data to recognize one or more objectsincluded therein based on at least one of a video analysis modelgenerated through deep learning based on channel metadata of a channelthat provides the video data and a separate video analysis modelreceived from the outside of the video analysis and management system.19. A computer-readable storage medium that stores a computer programfor integrated analysis and management of video data, wherein thecomputer program includes one or more instructions to be executed by oneor more computing devices in a video analysis and management system, andthe one or more instructions include: at least one instruction toreceive video data from any one channel of an input video providingdevice including a plurality of channels; at least one instruction toallocate the received video data to a specific video analyzing server ofa video analyzing server cluster including a plurality of videoanalyzing servers in the video analysis and management system; at leastone instruction to analyze the video data by the specific videoanalyzing server to recognize one or more objects included therein; andat least one instruction to generate a graphic user interface in apredetermined format based on an analysis result and an objectrecognition result.
 20. The computer-readable storage medium of claim19, wherein the one or more instructions include: at least oneinstruction to receive a search query for a specific object from anexternal user device; at least one instruction to search at least one ofa metadata database and a video database based on the received searchquery; and at least one instruction to transmit a search result to theexternal user device.